In a brightly lit clean room at the US Department of Energy’s SLAC National Accelerator Laboratory, engineers are building a car-sized digital camera for the Large Synoptic Survey Telescope.
When it’s ready, LSST will image almost all of the sky visible from its vantage point on a Chilean mountain, Cerro Pachón, every few nights for a decade to make an astronomical movie of unprecedented proportions.
The camera is a combination of many extremes. Its largest lens is one of the biggest ever created for astronomy and astrophysics. The ceramic grid that will hold its imaging sensors is so flat that no feature larger than a human red blood cell sticks up from its surface. The electronics that control the sensors are customized to fit in a very tight space and use as little power as possible.
All of these specifications are vital for letting LSST achieve its scientific goals. And not many of them are easy to achieve. The LSST camera will do what no camera has been capable of doing before, and building it requires solving technical problems that have never been solved before.
A game of ‘Operation’
“When you consider a project this complex, you can’t just dive in and say ‘Here, I’m going to design and build this in one shot,’ right?” says Tim Bond, head of the LSST Camera Integration and Test team at SLAC. “You have to divide and conquer. So you break it up into smaller pieces that individual groups can work on.”
One of those pieces is figuring out how to get the camera’s sensors into place.
The 3.2-billion-pixel LSST camera will be the largest digital camera ever constructed. Much like handheld digital cameras, the LSST camera will be made up of imaging sensors called charge-coupled devices—189 of them. These sensors and their bundles of electronics are arranged into 21 nine-sensor pallets called “rafts.” Each one weighs more than 20 pounds and stands almost 2 feet tall.
Each sensor is fragile enough to chip if it even touches one of the other rafts. And, to minimize gaps in the sensors’ images, all of the rafts must be installed two hundredths of an inch apart inside the camera’s ceramic grid.
The LSST engineers couldn’t possibly install the delicate rafts by hand without destroying them, so they took on the challenge of creating a device that could do this very specific task in their place.
They concocted one concept after another. Travis Lange, a SLAC mechanical engineer, created computer models of each to find a design that could both do the job and be built with the level of machining precision available.
“One of the bigger challenges for this is just the tolerance of all the individual pieces and how it corresponds to how much motion I am allowed to use,” Lange says. If a part is the wrong size by even just the width of a human hair, it’s a problem. "If you have many of those parts that are off by that much, those errors all stack up.”
One of the designs that the team drew up resembled a claw-machine game. The device would sit on a structure above the cryostat, the apparatus that keeps the camera cold. With a long arm, it would reach through to a raft waiting for installation below. Over the course of several hours, it would pull the raft up through a very precisely sized slot and into place in the grid.
Four specialized cameras pointing at the edges of the imaging sensors would help steer the raft into place without hitting neighboring sensors, and unique imaging software would measure the gaps between rafts in real time. “It’s a crazy game of ‘Operation,’” Lange says.
The team went with the claw-machine plan. In May 2018, they put it to the test with its first practice raft and a mock-up of the camera. After most of a day had passed, the raft was successfully in place.
The installation robot has since gone through several other successful test runs. Now that they’ve figured out the kinks in the process, installing each raft takes about two hours. Engineers plan to start the real installation process this summer.
Not your everyday refrigerator
The electronics and sensors crammed together inside the camera heat up as electricity runs through them. But heat is the enemy of astronomical observation. A warm sensor will sabotage its own observations by behaving as if it senses light where there is none. And as anyone who has ever heard their laptop fan working overtime before the computer crashed may know, heat can also cause electronics to stop working.
To keep the camera cold enough, the engineers needed to create a customized refrigeration system. They eventually made a system of eight refrigeration circuits—two for the electronics and six for the sensors.
Each of these systems works similarly to a kitchen refrigerator, in which a fluid refrigerant carries heat away from the object or area it’s supposed to cool. Networks of tubes carry the refrigerant into and out of the camera.
At first, the team used only metal tubes for this job. Metal is good at keeping moisture out, which is important because any water that gets into the tubes from the surrounding air would freeze and clog the system. At parts of the system where the camera would need to move around with the telescope as it points to different parts of the sky, the tubes were corrugated to make them into flexible metal hoses.
But there was a problem. The refrigeration system’s compressor, a device that forces the refrigerant to dump its absorbed heat outside the camera, uses lubricating oil to work smoothly. As the refrigeration system ran, some of the oil would leave the compressor and travel through the tubes.
This wouldn’t have been a problem if the oil had traveled at a consistent pace all the way through the circuit, back to the compressor. But that wasn’t happening; the oil was getting slowed down and sometimes trapped by the grooves in the corrugated metal hoses. The compressor was getting oil back in trickles or spurts rather than in a steady stream. This made the refrigeration system unpredictable and harder to maintain.
So the team switched to a different kind of hose for the refrigeration system’s “joints,” says Diane Hascall, a SLAC mechanical engineer on the LSST camera team. “You can almost think of it like a garden hose. But it’s a very special garden hose that’s made to work with refrigerants.”
The new hoses, called smooth-bore hoses, are made of layers of rubber, braid and other flexible materials, and they are smooth on the inside. The smooth hose lets oil return to the compressor more effectively, Hascall says.
But there was a trade-off. Unlike the metal hoses, the smooth-bore hoses do let some moisture in.
To deal with that, the team installed filter dryers that absorb moisture from the system. They are still figuring out how often the dryers need to be replaced to keep the camera in good shape.
Building next-gen technology
Building each component of a piece of technology as sophisticated as LSST is a challenge in itself, but the challenges don’t end there. Engineers must also design specialized equipment, software and procedures to test different pieces; put the pieces together; and determine what maintenance the technology will need to run smoothly.
“There’s a huge number of subsystems,” Bond says. “All of those subsystems have to present their products. And all those products have to be assembled and tested and work in the final finished full product.”
Bond says working on the project has been a great boon to the engineering team. He says figuring out all of the unexpected challenges that have come with making such an advanced piece of technology has been a great experience, and he looks forward to seeing what future projects the team will tackle together.
“It’s like picking a bunch of players to set up a hockey team or something,” Bond says. “We’ve actually put together a very good team, and we’re just getting some of our younger people up to speed and trained for the next generation of experiments and projects that will come along.”
